Linear Time Model Selection for Mixture of Heterogeneous Components

Our main contribution is to propose a novel model selection methodology, expectation minimization of information criterion (EMIC). EMIC makes a significant impact on the combinatorial scalability issue pertaining to the model selection for mixture models having types of components. A goal of such problems is to optimize types of components as well as the number of components. One key idea in EMIC is to iterate calculations of the posterior of latent variables and minimization of expected value of information criterion of both observed data and latent variables. This enables EMIC to compute the optimal model in linear time with respect to both the number of components and the number of available types of components despite the fact that the number of model candidates exponentially increases with the numbers. We prove that EMIC is compliant with some information criteria and enjoys their statistical benefits.